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[57] ABSTRACT 

A hyperspectral imager includes a focal plane having an 
array of spaced image recording pixels receiving light 
from a scene moving relative to the focal plane in a 
longitudinal direction, the recording pixels being trans- 
portable at a controllable rate in the focal plane in the 
longitudinal direction, an electronic shutter for adjust- 
ing an exposure time of the focal plane, whereby re- 
cording pixels in an active area of the focal plane are 
removed therefrom and stored upon expiration of the 
exposure time, an electronic spectral filter for selecting 
a spectral band of light received by the focal plane from 
the scene during each exposure time and an electronic 
controller connected to the focal plane, to the elec- 
tronic shutter and to the electronic spectral filter for 
controlling (a) the controllable rate at which the re- 
cording is transported in the longitudinal direction, (b) 
the exposure time and (c) the spectral band so as to 
record a selected portion of the scene through M spec- 
tral bands with a respective exposure time X q for each 
respective spectral band q. 

33 Claims, 8 Drawing Sheets 
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PROGRAMMABLE HYPERSPECTRAL IMAGE 
MAPPER WITH ON-ARRAY PROCESSING 

ORIGIN OF THE INVENTION 5 

The invention described herein was made in the per- 
formance of work under a NASA contract, and is sub- 
ject to the provisions of Public Law 96-517 (35 USC 
202) in which the Contractor has elected not to retain 
title. 10 

BACKGROUND OF THE INVENTION 

1. Technical Field 

The invention relates to spectral imaging devices 
useful in flight vehicles including space vehicles for I 5 
acquiring color images of a ground surface. 

2. Background Art 

Various imaging techniques are well-known in the 
art. U.S. Pat. No. 4,514,767 discloses a color filter with 
a computer-controlled imaging device which capable of 20 
synchronizing the selection of the color filter with the 
scrolling of the imaging device. U.S. Pat. Nos. 
4,827,528; 4,747,157; 4,551,768 and 4,398,256 disclose 
the technique of co-adding image pixels to enhance 
signal to noise ratio, but are not related to spectral filter- 25 
ing. U.S. Pat. Nos. 4,947,446; 4,885,787; 4,574,357; 
4,703,513; 4,330,833; 4,707,859 and 4,724,544 disclose 
various electronic digital image filtering techniques not 
related to pixel co-adding or spectral filtering. U.S. Pat. 
Nos. 5,023,921; 4,694,342 and 4,731,864 disclose various 30 
spatial filtering techniques not related to pixel co- 
adding or spectral filtering. U.S. Pat. No. 4,792,231 
discloses a laser speckle imaging technique for satellites 
not related to spectral filtering or pixel co-adding. 

During the 1970’s, NASA launched a number of 35 
LANDSAT spacecraft equipped with multispectral 
sensors. These sensors, equipped with between four and 
seven simultaneous spectral channels, mapped the sur- 
face of earth at ground resolutions of approximately 30 
m. They have been applied to a great variety of investi- 40 
gations of the earth’s surface, atmosphere and bio- 
sphere. In the 1980’s, it has been France that has lead in 
the further development of orbital multispectral sens- 
ing. Imagery acquired with the SPOT satellite has a 
comparable number of spectral channels to LAND- 45 
SAT, but the surface resolution has been improved. 

During the 1980’s, NASA has been pioneering the 
development of hyperspectral sensors which would De 
capable of mapping the surface of the earth in more than 
a hundred simultaneous spectral channels. With this 50 
heightened degree of spectral characterization, hyper- 
spectral sensing enables identification of surface materi- 
als and much more definitive characterization of earth 
surface processes than is possible with multispectral 
sensing. However, the hyperspectral sensing technol- 55 
ogy presents some formidable instrumental, communi- 
cations and data processing problems. 

IMAGING SPECTROMETER TECHNOLOGY 

The instrumentation technology used to implement 60 
hyperspectral sensing to date has been quite different 
than that used in multispectral sensing. In the hyper- 
spectral imaging concepts proposed to date, such as in 
U.S. Pat. No. 4,497,540 to Breckinridge et al., the scene 
has been imaged on a spatially confining aperture or slit 65 
and dispersed to illuminate a detector array using a 
diffraction grating or prism. In this way, all spectral 
channels are acquired simultaneously and the field of 
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view in each spectral channel is precisely the same. This 
is the key feature of “imaging spectromentry” which 
can be contrasted with “spectral imaging” in which 
different spectral channels are acquired sequentially in 
time and are not inherently spatially registered to one 
another. 

Although imaging spectrometry is currently the tech- 
nology of choice for hyperspectral sensing it has limita- 
tions. Let us consider the HIRIS instrument which is 
designed to map the earth’s surface from 800 km alti- 
tude at 30 m spatial resolution. HIRIS images the 
Earth’s surface on a narrow slit and then disperses the 
radiation transmitted or reflected from the slit such that 
it illuminates a two dimensional focal plane array 
(FPA). The FPA is scanned and read out each time the 
scene advances by one pixel width (equivalent to the slit 
width and the size of a spatial resolution element on the 
ground) by virtue of the orbital motion of the space- 
craft. 

Since the orbital motion of a spacecraft is constrained 
by the laws of celestial dynamics, this mode of opera- 
tion renders the imaging spectrometer a rather inflexi- 
ble instrument. At the HIRIS altitude of 850 km., the 
exposure time is 3.9 ms and for some scenes of low 
reflectivity the signal levels are already very low. In 
particular, the exposure time will inevitably decrease 
with increasing spatial resolution and further degrade 
signal-to-noise. The HIRIS already seems to be ap- 
proaching a practical limit of 30 meters. In additional, 
the FPA must be read more rapidly in order to avoid 
spatially smearing the image. In practice this severely 
limits the applicability of the imaging spectrometry 
techniques to fairly coarse surface resolution and results 
in large and expensive instruments to achieve maximum 
optical throughput. 

The technique of acquiring multispectral mutually- 
registered, high signal-to-noise spectral images of sta- 
tionary or slowly-moving targets by viewing the scene 
through a spectral filter is well established. It has been 
implemented on a number of space missions using 
stepped mechanical filters wheels. The limitations of the 
technique are (1) the number of filters that can be practi- 
cally employed in such a system, (2) the speed at which 
they can be changed and (3) the limitation to sequential 
access. 

SUMMARY OF THE INVENTION 

Motivated by the foregoing limitations of imaging 
spectrometer technology, and by the recent emergence 
of new concepts in electrically programmable high 
resolution spectrally agile filters (SAF’s), the potential 
of hyperspectral imagers has been greatly expanded in 
the present invention. The present invention is a Pro- 
grammable Hyperspectral Imaging Mapper with on 
Array Processing (PHIMAP). The enhanced capabili- 
ties of PHIMAP reside not only in the SAF but also in 
the exploitation of signal processing on the focal plane 
array. The present invention exploits the development 
of electrically-addressable SAF’s based on Acousto 
Optic Tunable Filter (AOTF) technology to solve all 
three of the problems or limitations of viewing the scene 
through a spectral filter using stepped mechanical filter 
wheels. As a result the present invention makes a true 
hyperspectral capability feasible. 

The specific application discussed here, however, is 
hyperspectral imaging from a platform in low earth 
orbit with rapid ground motion. Hitherto, SAF’s have 
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not been thought to be useful in imaging for land remote 
sensing. As we shall show, there are methods of exploit- 
ing spectral agility provided they are used in conjunc- 
tion with signal processing on the focal plane and they 
conifer a number of advantages over other techniques 5 
of acquiring hyperspectral data. 

The invention synchronizes changes in spectral pass 
band with frame transfers of an imager such as a focal 
plane array (FPA). Different spectral bands (colors) 
may be given different exposure times, depending upon io 
the desired degree signal quality enhancement for indi- 
vidual spectral bands. The detected image is transferred 
in the FPA along the direction of motion with respect 
to the scene in synchronism therewith so that successive 
pixels of the FPA view the same scene elements succes- 15 
sively and their outputs are co-added to enhance signal- 
to-noise. Rather than permitting slit width to dictate a 
fixed exposure time, the invention permits rapid change 
in exposure time between spectral bands by electroni- 
cally shuttering the FPA. 20 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a simplified block diagram illustrating sys- 
tem embodying the invention. 

FIGS. 2 a through 2c illustrate prior art techniques for 25 
performing conventional image motion compensation 
for three different exposure times. 

FIGS. 2d through 2/illustrate image motion compen- 
sation or pixel co-adding for exposure times corre- 
sponding to FIGS. 2a through 2c, respectively. 30 

FIG. 3 illustrates how to form contiguous image 
strips from the results of the method of FIGS. 2d 
through 2/ 

FIG. 4 a and FIG. 4 b illustrate the method of FIG. 3 
applied to focal plane arrays having discrete photosensi- 35 
tive elements. 

FIGS. 5a and 5b comprise a diagram illustrating focal 
plane array apparatus employed in carrying out the 
invention. 

FIG. 6a illustrates a first mode of the invention for ^ 
operating the apparatus of FIG. 5. 

FIG. 6b illustrates a second mode of the invention for 
operating the apparatus of FIG. 5 in which image mo- 
tion compensation is performed. 

DETAILED DESCRIPTION OF THE 45 
INVENTION 

I. CONCEPT 

The imaging geometry is illustrated in FIG. 1. An 
image or scene of the surface of the Earth (for example) 50 
is focussed by imaging foreoptics 102 through a Spec- 
trally Agile Filter (SAF) 104 such as an AOTF on to 
the surface of an imaging device 106 . The foreoptics 
102 , the SAF 104 and the imaging device 106 are all 
mounted on an airborne or orbiting platform which is 55 
moving relative to the scene 100 . The imaging device 
must possess three functions: registering the input of 
optical radiation, storage of this information in some 
form that preserves its spatial character, and transmittal 
or readout of the information for display. In some me- 60 
dia, certain of these functions may be subsumed in the 
same part of the device. In this section, we consider the 
most general form of the invention when the imaging 
device is either a continuous recording medium such as 
photographic film or a pixellated device or focal plane 65 
array (FPA) comprised of an array of discrete detec- 
tors. In section II, we consider in more detail the real- 
ization with a focal plane array. Then in section III we 
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consider a specific form of the invention using a particu- 
lar kind of focal plane array, a Charge Coupled Device 
(CCD). 

Image motion compensation is an element of the 
preferred embodiment of invention. Here, we consider 
first the general concepts of image motion compensa- 
tion, how a conventional image motion compensation 
system is implemented, and then how image motion 
compensation is implemented in the present invention. 
Finally, we describe how rapid spectral pass band 
changes, obtained with the SAF, can be used with a 
single imaging device to obtain multiple mutually-regis- 
tered strips of motion-compensated images in which 
each strip is acquired through a different spectral filter. 

A. Image Motion Compensation — General 

When high resolution cameras are used to obtain a 
near vertical view of the surface of the earth (or another 
planet) from an aircraft or a spacecraft in a low altitude 
orbit, the optical image is not stationary in the focal 
plane but moves rapidly across it because of the motion 
of the platform relative to the ground. To acquire a 
blur-free recorded image, one approach is to expose the 
sensor for a time short with respect to the time taken for 
the optical image to move by a resolution element. 
However, if the scene luminance is low, a situation 
which is exacerbated using narrow spectral bands, in- 
sufficient light is collected during the exposure time and 
the resulting image has low signal-to-noise. Image mo- 
tion compensation is any scheme for causing the photo- 
detection and/or storage medium to be transported at 
the same speed as the optical image thereby allowing 
much larger exposure times. 

The PHIMAP concept uses a form of image motion 
compensation which is quite different from conven- 
tional systems. To set the proper context for the discus- 
sion of the PHIMAP implementation, a conventional 
image motion compensation system is first described. 

B. Conventional Image motion Compensation 

A conventional image motion compensation system 
consists of a focal plane aperture slit which limits the 
area of the focal plane over which light can reach the 
photo-recording medium. As a result of spacecraft mo- 
tion, the scene moves at a uniform speed across the slit. 
The photo-recording and storage medium behind the 
slit is transported either mechanically or electronically 
at the same rate as the scene to compensate for image 
motion. 

The process is illustrated in FIGS. 2 a through 2c for 
10 ms, 20 ms and 50 ms exposure times, respectively. 
These require slits of progressively greater width (along 
track dimension) indicated in solid line. FIGS. 2a- 
through 2c are graphs plotting the exposure time as a 
function of position along the flight line. Before the 
scene passes the slit, the exposure is zero; as it crosses 
the slit it increases linearly to a maximum of either 10, 
20 or 50 ms; and beyond the slit it maintains a constant 
value. 

This approach works very well with a single sensor 
for monochromatic imaging. It is implemented photo- 
graphically by transporting film from one spool to an- 
other under the slit. It can also be implemented elec- 
tronically with a “bucket-brigade type” focal plane 
array such as a charge coupled device (CCD). Charge is 
clocked into a temporary storage register at the edge of 
the slit from where it is read out and recorded on a 
sequential recording medium such as magnetic tape. In 
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either film or electronic realization, a continuous image 
of uniform exposure is produced. 

C. Image motion compensation with PHIMAP 

Image motion using the PHIMAP concept of the 

present invention has some important differences from 5 
the conventional approach. First, the same fixed aper- 
ture slit is used for all exposure times. A high speed 
electronic shuttering and readout mechanism replace 
the uniform transport of the photosensing and storage 
medium behind the slit. The purpose is to allow infor- io 
mation for the same part of the surface to be recorded in 
several spectral bands using a single focal plane array. 

The process is illustrated in FIGS. 2d through If for 
10 ms, and 50 ms exposure times. As indicated in FIGS. 

2d through 2 f the same slit width is used for each of the 15 
three different exposure times. The exposure time is 
varied by controlling the time during which the photo- 
sensitive surface is activated. At the end of that time, 
the stored image is transported rapidly to a masked-off 
area for read out. 20 

The exposure time (dotted line) is plotted as function 
of distance along the flight line at the conclusion of the 
integration process. Note that there is a uniformly illu- 
minated or exposed part of the image segment which is 
narrower than the slit. Beyond the uniformly exposed 25 
part, exposure ramps down linearly both up-track and 
down-track. Note also what happens when the exposure 
time is increased. It is clear that the exposure level can 
be increased but only at the expense of making the re- 
gion of uniform exposure narrower. 30 

D. Conditions for complete image strips in a PHIMAP 
implementation. 

One can conceive of ways of reconstructing images 
from mosaics using both the uniformly illuminated 
image segments and the ramps on either side of the 35 
uniformly exposed regions of FIGS. 2 c-2f However, in 
most applications it is simpler to discard the ramps and 
to use a wider slit which exposes a larger photosensitive 
area to minimize the performance consequences of dis- 
carding the ramps. 40 

The limitations on the sizes of the exposures that can 
be obtained are illustrated in FIG. 3. Here, nine image 
frames are projected on to a map base. Successive 
frames have been offset slightly from one another and 
frames acquired with different spectral filters are shown 45 
separately in order to show more clearly the spacings 
along the flight path. The intensity ramp on the leading 
edge of the frame is shaded; the symmetrical intensity 
ramp on the trailing ramp is omitted for clarity. 

Any object will remain in the field of view for 100 5Q 
milliseconds. As a result the cycle of running through 
filters must certainly be completed in less than 100 milli- 
seconds to avoid gaps. The exposure sequence in this 
case is: 


Red Filter 10 milliseconds 

Green Filter 20 milliseconds 

Blue Filter 50 milliseconds 


which leaves 20 milliseconds to spare. 

A general condition for the performance limits of this 
kind of imaging system with q filter selections is: 

+t q )^Z (l) 

where ti, t2, . . . , t q are exposure times in successive 
spectral bands and T is the time required for an object to 
traverse the field of view. In principle, q can be arbitrar- 
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ily large, and is limited primarily by the switching speed 
of the SAF. 

This relationship only applies, however, if it is possi- 
ble to recover signal from the ramp regions. If only the 
regions of uniform exposure are used, then the image 
portions corresponding to shaded areas of FIG. 3 must 
be discarded. In this case, the red frames still have over- 
lap, the green frames just provide complete coverage 
but the blue frames have substantial gaps. 

A general condition for the performance of this kind 
of imaging system with q filter selections is: 

(«l + <2+ - ■ . +tgWr— MAXfo, ft, ... . tq]) (2) 

where “MAX[ti, t2, • • . , %]” is the maximum value of 

« » • > t q. 

An alternative to recovering image data from the 
exposure ramps (shaded regions) would be to use a 
larger focal plane array. The right-most frame grouping 
in FIG. 3 (labelled 3', 6', 9') illustrates the sequence for 
the blue channel. This preferred sequence uses an FPA 
for which an object takes 130 milliseconds to traverse 
the field of view. The preferred sequence provides con- 
tiguous coverage (no overlap/no gaps) for the blue 
images with exposure time 50 milliseconds and overlap 
for both the red and the green. 

E. Relationship to a Discrete Focal Plane 

In an implementation of the above concept using a 
discrete focal plane array, time in milliseconds trans- 
lates into pixel steps measured along track. For exam- 
ple, if the scene advances by one pixel in a millisecond 
then the image motion compensation for red, green and 
blue respectively would be implemented by 10, 20 and 
50 shift operations respectively. 

II. IMPLEMENTATION WITH A FOCAL PLANE 
ARRAY 

The imaging geometry is again as illustrated in FIG. 
1. An image or scene 100 of the surface of the earth (for 
example) is focussed by imaging foreoptics 102 through 
an SAF 104 such as an AOTF on to the surface of a 
focal plane array (FPA) 106 having L columns and N 
rows of picture elements or pixels. The height of one 
frame is understood to be N pixels in this example. The 
foreoptics 102, the SAF 104 and the FPA 106 are all 
mounted on an airborne or orbiting platform moving 
relatively to the scene 100. For simplicity, the orienta- 
tion of the foreoptics 102 and FPA 106 is such that the 
direction of motion is parallel to columns of pixels in the 
FPA 106. 

Freeing ourselves for a moment from preconceptions 
about the kinds of operations that can be performed on 
this moving image, let us establish the various ways in 
which information can be recorded on an array of dis- 
crete pixels. For simplicity we will restrict consider- 
ation to viewing near the nadir where scale differences 
across the array are negligible. 

One mode of operation might be to change the spec- 
tral passband each time the scene 100 advances by a 
pixel separation (i.e., the distance between adjacent 
pixel centers in a column of the FPA 106) along the 
ground track. Up to N spectral channels of data (the 
number of pixel rows) each with a maximum exposure 
of dT, the time in which the image of the scene ad- 
vances by a pixel separation along the ground track, can 
be acquired before the scene advances the width of the 
frame. By cycling through the N spectral channels, a 



5,379,065 


7 

series of immediately contiguous frames of the surface 
of the earth are acquired which form a continuous strip 
map along the ground track. This operation is illus- 
trated for two cycles, each cycle consisting of five 
frames (labelled “1”, “2”, etc. through “10”) in FIG. 4 a 5 
for an array with N equal to 5. Note that with the dis- 
crete array, the maximum number of filter selections for 
continuous coverage is N, in accordance with the illus- 
tration of FIG. 4a. 

Another mode, useful in circumstances where the 10 
light exposures are restricted by low radiance, narrow 
filter bandwidth and/or rapid frame times, is to admit 
only a single spectral band and co-add the signal to 
improve signal-to-noise. In effect, this provides elec- 
tronic image motion compensation (IMC). The net ex- 15 
posure time for each resolution element is increased 
from dT to N X dT, thereby improving image signal-to- 
noise. This mode of operation is also illustrated in FIG. 

4a. 

The most powerful embodiment of the present inven- 20 
tion, however, is a combination of spectral bandpass 
variation and image motion compensation. In this case, 
the number of spectral channels chosen and the integra- 
tion times for contiguous strip coverage must satisfy the 
following inequality: 25 

(«i+«2+ • - * +nq-i+n g )£N (3) 

where m, . . . n^are the multiples of the exposure time 
dT for each of the spectral channels or bands 1 through 30 
q. The relationship with inequality (1) will be quite 
evident. (However, note that ni, nz, . . . , n q .\, n q only 
take the discrete values 1, 2, 3, ... , etc., whereas t 2 , . . 

. , t q vary continuously.) This option is illustrated in 
FIG. 4b for nine consecutive frames labelled “1”, “2”, 35 
etc., again using an array with N= 10. In this example, 
only three spectral bands are used. The particular selec- 
tion of ni, n 2 , n 3 illustrated here is ni=l, n 2=2 and 
n 3 = 5 satisfying the inequality (3). 

Although in the illustrations of FIGS. 4a and 4b, N is 40 
shown with a very small value, values of N=100 to 
1000 are preferred. This provides the possibility of si- 
multaneously providing a large number of spectral 
channels and substantial exposure time multiples. 

A number of advantages are evident for this tech- 45 
nique for the PHIMAP technique for acquiring hyper- 
spectral data: 

1. Relative to the imaging spectrometry technique of 
U.S. Pat. No. 4,497,540, it provides a capability for 
image motion compensation. It also enables the 50 
instrument user to flexibly allocate the resource of 
the number of focal plane array rows to provide 
larger exposure time multiples for spectral chan- 
nels with the lowest radiance. In this way, im- 
provement in signal-to-noise ratio and the number 55 
of spectral channels can be traded against one an- 
other under the constraint of the inequality (1). 

2. By implementing image motion compensation 
(IMC) and related signal processing operations on 
the focal plane array, the rate and volume of digital 60 
data that must be handled by the spacecraft science 
data management system can be drastically re- 
duced. 

3. By programming both the SAF 104 and the FPA 
106 with spectral “matched filters”, specific min- 65 
eral or vegetative types can be identified by analog 
operations implemented within the instrument 
without requiring an auxiliary signal processor. 
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4. Adaptive modes of image acquisition are feasible in 
which the data acquired in one frame can trigger 
more detailed and comprehensive spectral cover- 
age during the same orbit. 

To achieve these capabilities, requires SAFs and 
FPAs with the speed of response to change spectral 
bands and perform the requisite image processing oper- 
ations at time scales comparable to or less than dT. It 
also requires that these devices have high throughput, 
superior radiometric fidelity and can maintain high 
spatial resolution simultaneously under these condi- 
tions. 

III. IMPLEMENTATION WITH A CHARGE 
COUPLED DEVICE 

The specific implementation considered here uses an 
Acousto Optical Tunable Filter (AOTF) for the SAF 
and a frame transfer silicon charge coupled device 
(CCD) for the focal plane array. This device would be 
useful for the spectral range from 0.1 to 1.0 p tm. 

A. Spectrally Agile Filter 

Referring now to FIG. 1, in the AOTF version of the 
SAF 104, the spectral pass band is changed by an RF 
driver 110 exciting an acoustic signal (at RF frequen- 
cies) in a crystal with strain-dependent refractive prop- 
erties. The spectral bandpass can be changed by modi- 
fying the frequency of excitation. 

The present generation of non-collinear AOTFs fab- 
ricated in tellurium oxide can provide throughput in 
excess of 50% in the spectral range from 0.5 to 5.0 pi m. 
Although existing devices do suffer from thermal-dis- 
tortion induced resolution loss at the highest levels of 
throughput, these effects are most serious in the mid 
range infrared. Wavelength-dependent translation of 
the image can be compensated either optically or elec- 
tronically. Sub-millisecond random access for spectral 
selections is quite feasible which satisfies requirement 
that spectral access time be small compared to dT. 

The PHIMAP could also be implemented with other 
forms of SAF’s. 

B. Focal Plane Array 

The specific implementation of PHIMAP examined 
here is illustrated in the diagram of FIGS. 5 a and 5b, 
and uses a fairly conventional frame transfer silicon 
CCD controlled by control electronics 112 which also 
control the SAF 104. Although the construction of the 
FPA 106 in the novel system of FIGS. 5a and 5b is 
conventional, the manner in which the device is electri- 
cally operated by the control electronics 112 is not. 

C. Spectral image acquisition: without image motion 
compensation 

In the mode of operation depicted in FIG. 6a, the 
imaging area of the CCD is exposed to light for the 
desired exposure time. At the completion of the expo- 
sure, the optically generated electric charge is rapidly 
transferred into the optically-shielded frame-store re- 
gion of the device by synchronously clocking the paral- 
lel gates P21, P22, P23, P31, P 32 > P33, in both the image 
and frame store areas respectively. Tkis transfer opera- 
tion can be executed in a few tens of microseconds 
which is usually short enough to eliminate image smear. 

During the next exposure of the CCD, the parallel 
gates in the frame store and image areas of the CCD are 
decoupled. Charge transfers are implemented in the 
optically-shielded frame store area, but not in the active 
image area, so that lines of image data are sequentially 
shifted into the serial gate and read out through an 
on-chip amplifier (not shown) and the signal, levels 
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digitized. This is accomplished by synchronously clock- 
ing the CCD gates P 31 , P 32 , P 33 , ^41, P 42 , P43- This 
readout process must be completed in time for the trans- 
fer of the next exposed image from the imaging arena. 

By using as much of this period as possible and minimiz- 5 
ing the readout bandwidth, the readout electronic noise 
is minimized. 

If the spectral band is changed synchronously with 
the frame transfer operation and the time between frame 
transfers (dT) is synchronized with the time required to 10 
advance by one pixel width along the ground track, we 
have a hyperspectral imaging capability of the form of 
FIG. 4 a. In principle, by not changing spectral bands at 
the time of each frame transfer, we can generate multi- 
ple sets of image data in the same spectral band which 15 
can be co-added off focal plane to improve signal-to- 
noise. However, as will now be discussed, of particular 
interest is an on-focal-plane scheme for performing this 
coadding operation before the readout amplifier and 
analog-to-digital conversion noise is added to the signal. 20 

D. Spectral image acquisition: with on-focal plane 
image motion compensation 

An image of the earth is focussed through the AOTF 
onto the CCD, which has an imaging section with L 
columns and N rows of detectors. The camera must be 25 
oriented with the direction of spacecraft motion parallel 
to the columns of pixels in the CCD. Because of space- 
craft motion, the image of the scene advances by one 
pixel along the ground track in a time dT. 

As the spacecraft advances by one pixel separation 30 
along the ground track, the parallel clock in the imaging 
area is used to scroll the FPA by one pixel spacing so 
that the electronic image remains registered with the 
optical image. This can be repeated several times until 
adequate signal accumulates. At the conclusion of n 35 
image shifts, a frame transfer operation takes place and 
is synchronized with a change of spectral band. This last 
operation distinguishes this sensor from a conventional 
CCD-based electronic Image Motion Compensation 
(IMC) system. Readout is initiated immediately after 40 
frame transfer. 

In the mode of operation illustrated in FIG. 6b, the 
coadding operation is performed in analog fashion on 
the focal plane array by using the parallel clocks P 21 , 
p22> P 23 controlling the image area to scroll the FPA 45 
106 by pixel spacing each time period dT that the mo- 
tion of the optical image of the earth surface across the 
array progresses by one pixel. This is illustrated in FIG. 

6b for exposures of dT, 2 dT, and 3 dT. At the conclu- 
sion of each exposure, a frame transfer operation takes 50 
place and is synchronized with a change of spectral 
band. Readout is initiated immediately after frame trans- 
fer. If the same readout rate is used as in conventional 
operation, then for anything more than a single expo- 
sure, readout will be completed in a fraction of a typical 55 
exposure time. 

With this implementation, the signal-to-noise of the 
image is improved by doubling or tripling the number 
of photogenerated electrons generated for each spatial 
and spectral resolution element. This is because the 60 
image motion compensation provides an effective expo- 
sure time of n X dT sec. A further gain can be realized 
by exploiting the longer exposure time to lengthen the 
readout time. This reduces the read noise, i.e. the uncer- 
tainty with which the signal levels are measured. 65 
Lengthening readout is practical if successive spectral 
bands have about the same value or gradually increas- 
ing values of n. Thus, there are two different ways in 
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which data quality is improved in this particular PHI- 
MAP implementation. Note that if the readout time is 
lengthened, the scrolling operation for image motion 
compensation must be synchronized with the much 
faster readout or scrolling operation used to read image 
data from the frame store area to the signal chain as 
illustrated in FIG. 6b. 

For n=10, a SNR gain of 30 is possible in the low 
signal read-noise-dominated regime; at higher signal 
levels, where shot noise or fixed pattern noise domi- 
nates, the SNR improvement decreases to a factor of 
three. At intermediate signal levels, the SNR improve- 
ment lies between these values. 

E. Registration and Mapping of images 

A long-recognized disadvantage of framing imagers 
for spectral mapping from an orbital spacecraft has been 
the data management and computational problems of 
registering frames within a swathe. Even when registra- 
tion can be performed, the typical fractional pixel off- 
sets cause spurious spectral signatures. This can be miti- 
gated by spatial resampling but this results in a loss of 
spatial resolution. Techniques for overcoming these 
problems are discussed here. 

Synchronization and alignment of images would be 
implemented by inserting “synch” frames at regular 
intervals in the imaging sequence. The synch frames are 
spaced such that surface features can be tracked 
through at least two and possibly more frames. Al- 
though ideally all the synch frames should be acquired 
through the same spectral filter, in practice because of 
the high correlations in brightness between spectral 
bands for most natural scenes, different spectral bands 
may work perfectly adequately. 

By cross-correlating areas containing high scene ac- 
tivity in two successive frames, fractional pixel mis- 
alignment across track between the synch frames and 
intervening frames can be eliminated. Correlation data 
would also be used to synchronize the image motion 
compensation with the along track motion. 

F. Comparison of PHIMAP-1 with “ideal” PHIMAP 
sensor 

The CCD implementation described here (PHIMAP- 
1) falls short of the performance of the ideal PHIMAP 
focal plane array characterized by the inequality (3) 
stated previously herein. Areas of exposed image which 
transfer in or out of the active frame area during an 
exposure because of the scrolling operation needed for 
image motion compensation, are not reconstructed into 
image data on the focal plane and must be discarded. It 
is possible to envisage focal plane architectures in 
which these data would be retrieved but it requires 
large amounts of on-focal plane storage. However, the 
loss is quite small except for the case of extreme 
amounts of image motion compensation. Taking this 
effect into account, PHIMAP-1 performs as if the imag- 
ing array height were reduced by the maximum IMC 
used in the imaging sequence by obeying the following 
requirement more stringent than the requirement of 
inequality (3): 


(«l+«24- ■ • • +rt ? )=[iV— MAX (/*1 — 1, ni~ 1, .... 

*<7-1)1, (4) 

(where “MAX(. . .)” denotes the maximum value of the 
set of quantities in parentheses), which is clearly analo- 
gous to inequality (2). 

A further reduction in effective imaging array height 
would occur if the frame transfer time were not short 
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compared with exposure times. This is only expected to 
become a factor with very large arrays used at very 
short dwell times. 

PERFORMANCE ANALYSIS 5 

There are three principal contributions to the noise 
that corrupts the measurement of intensity differences 
with a focal plane array: read noise, shot noise and fixed 
pattern noise. Read noise is independent of the intensity 
of the signal being observed and is a characteristic of 10 
the sensing system. Shot noise is a result of the statistical 
character of the photon field and depends on the square 
root of signal intensity. Fixed Pattern Noise is directly 
proportional to signal intensity. Because of these depen- 
dencies, read noise dominates at small signal levels, shot 15 
noise is most important at moderate intensities and fixed 
pattern noise is most important at high intensities. Noise 
contributions from temporal variability in calibration 
and non-linearity which are important in some types of 
infrared detector are not considered here. 20 

Implementation of a PHIMAP using a frame-transfer 
charge coupled device has the effect of increasing the 
signal registered on the detector by an amount n, where 
n is the number of pixels of image motion compensation 
used for that particular spectral band. For instance if 25 
n= 10 then the signal recorded by the detector will 
increase by a factor of 10. However, to gauge the im- 
pact on overall performance or signal-to-noise, it is 
necessary to consider the impact on the three primary 
contributors to noise. 30 

a) Read noise is reduced because PHIMAP allows the 
time available for readout of the detector to be 
increased. In high performance CCD’s, the read 
noise depends on the square root of the bandwidth. 
Operating with n= 10 can reduce the read noise by 35 
a factor of 3.16 in the high speed regime where 
these systems operate. 

b) Shot noise decreases relative to the signal because 
with n= 10, signal is increased by a factor of 10 but 
shot noise depending on the square root of signal 40 
only increases by a factor of 3.16. 

c) Fixed Pattern Noise decreases relative to the signal 

provided that the fixed pattern noise approximates 
a random distribution which is approximately satis- 
fied for many sensors. In the case of n=10, the 45 
mean response of a group of 10 pixels will have a 
variance which is a factor of 3.16 lower than a 
single pixel. As a result, when n=10, the signal 
increases by a factor of 10, but the fixed pattern 
noise will only increase by a factor of 3.16. 50 

IV. SPECTRAL CAPABILITIES OF PHIMAP-1 

A. Spectral Equalization 

It is characteristic of natural scenes, that different 
spectral ranges can have widely different brightnesses. 55 
For example, in the 1 to 2.5 pm spectral band, the 
brightness varies by almost two orders of magnitude as 
a result of the roll-off in the spectral brightness of the 
illuminating solar radiation. As noted earlier, imaging 
spectrometers do not permit the exposure in each spec- 60 
tral band to be varied. The exposure in each spectral 
band is the same and equal to the signal pixel dwell time, 
i.e. the time the spacecraft takes to advance a signal 
pixel width along the ground track. In PHIMAP, on the 
other hand, the total exposure time available for all 65 
channels is the time the spacecraft takes to advance of 
an entire frame width along the ground track. With 
frame widths of several hundred pixels, this exposure 
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time is large and can be flexibly allocated between spec- 
tral bands. An example of how this might be imple- 
mented in the 1 to 2.5 p, m region is illustrated in FIG. 
6b. 


B. Spectral Bandpass Control 

The PHIMAP-1 has the capability for additively 
combining signals from a number of spectral bands on 
the FPA. This allows a complex spectral passband to be 
synthesized and changed adaptively during the mission. 
By tailoring the spectral passband to optimize discrimi- 
nation of those spectral signatures to be mapped, it 
enables information to be acquired with FPA data rates 
that are much lower than for the equivalent imaging 
spectrometer. 

There are several ways in which spectral bandpass 
control may be implemented. We will focus on only 
two here: 

a) Sub Pixel Dwell Time Spectral Scanning: In this 
mode, the AOTF is programmed to sweep out a 
spectral bandpass during the pixel dwell time. This 
provides a great deal of flexibility in allocating the 
frame time to complex family of matched filters. 
However, the recruited sweep speeds may exceed 
those conveniently available. More seriously, any 
chromatic image shift cannot be corrected. 

b) Spectral Scanning Synchronized with Pixel Step- 
ping: In this mode, the spectral band changes are 
synchronized with Pixel Shifts. This has the advan- 
tage that any chromatic shift could in principle be 
corrected by image scrolling. Its disadvantage is 
that the number of steps available in the IMC mode 
is likely to be much less than the total number of 
spectral channels that are desired. 

C. Adaptive Signal Processing 

Assembly of strip images and mutual registration of 
spectral information was described hereinabove with 
reference to FIGS. 2d-2f and FIG. 3. Comparison of a 
pair of matched filtered images can then be performed 
to greatly enhance compositional discrimination rela- 
tive to what is possible with a single frame. 

By performing real time processing of these signa- 
tures it should be possible to adaptively acquire data. 
For instance, the first part of a frame time could be 
allocated to a filters searching for a particular signature. 
If the signature appeared, this might be used to initiate 
storage of data or even acquisition of higher resolution 
data during the remainder of the frame time. 


V. APPLICATION TO INFRARED 
HYPERSPECTRAL IMAGING 

Most of the above considerations apply to applica- 
tions of the PHIMAP technology in the “silicon” spec- 
tral range i.e. from 0.1 to 1.0 pm. The spectral range 
from 1 to 2.5 pm is the other key spectral range that 
must be examined for this technology. 

A. AOTF capabilities 

Tellurium oxide has high transmission from 0.5 to 5.2 
jitm and these AOTFs have been demonstrated through- 
out this spectral range. 

B. Focal Plane Array Technology 

At present there is no practical monolithic infrared 
CCD technology in which detection and transfer of 
charge takes place in the same semiconductor. Conse- 
quently, a straightforward adaptation of the PHIMAP 
concept described above for the “silicon” spectral re- 
gion is not possible. Here we consider some alterna- 
tives: 

1. Mercury Cadmium Telluride Hybrid Arrays 
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In an early form of this technology, a “direct injec- 
tion” circuit is used to inject charge from a Mercury 
Cadmium telluride diode into the well of a silicon CCD. 
The simplest concept for an infrared PHIMAP is to 
build a hybrid direct injection array using a frame trans- 5 
fer CCD. 

2. Platinum Silicide Schottky Barrier CCD. 

This focal plane array technology has a number of 
advantages: it is monolithic and highly producible, and 
these devices can approach the uniformity of a silicon 10 
VNIR CCD and very low noise operation is possible in 
the CCD architecture. 3. Z Plane technology 

The application of Z plane technology focal planes to 
PHIMAP sensors may be carried out by the skilled 
worker in accordance with the foregoing. 15 

While the invention has been described in detail by 
specific reference to preferred embodiments, it is under- 
stood that variations and modifications thereof may be 
made without departing from the true spirit and scope 
of the invention. 20 

What is claimed is: 

1. AA hyperspectral imager, comprising: 

focal plane means comprising an array of spaced pixel 
recording means for receiving light from a scene 
moving relative to said focal plane means in a Ion- 25 
gitudinal direction, said pixel recording means 
being transportable at a controllable rate in said 
focal plane means in said longitudinal direction by 
rate-controllable transport means; 

controllable electronic shutter means for adjusting an 30 
exposure time tq of said focal plane means, 
whereby pixel recording means in an active area of 
said focal plane means are removed therefrom and 
signal charges accumulated therein are stored upon 
expiration of said exposure time; 35 

controllable electronic spectral filter means for se- 
lecting a spectral band q of light received by said 
focal plane means from said scene for each expo- 
sure time tq; and 

electronic controller means connected to said trans- 40 
port means, to said electronic shutter means and to 
said electronic spectral filter means for controlling 

(a) said controllable rate at which said recording 
means is transported in said longitudinal direction, 

(b) said exposure time tq and (c) said spectral band 45 

q so as to record in respective ones of said pixel 
recording means a selected portion of said scene 
through M spectral bands with a respective expo- 
sure time tq for each respective spectral band q, 
wherein M is a positive integer. 50 

2. The hyperspectral imager of claim 1 wherein dur- 
ing said exposure time tq each pixel recording element 
is transported by m pixel spacings in said longitudinal 
direction while said scene advances in said focal plane 
means by m pixel spacings in said longitudinal direction, 55 
whereby pixel values of said scene as recorded in said 
pixel recording means are co-added m times so as to 
enhance a signal-to-noise ratio thereof, and wherein m is 

a positive integer. 

3. The hyperspectral imager of claim 2 wherein X q , M 60 
and m are each different numbers for different values of 

q, whereby different scene portions are recorded in 
different groups of said pixel recording means with a 
respective exposure time t q from each band q and co- 
added a different number of times m for each respective 65 
spectral band q. 

4. The hyperspectral imager of claim 2 wherein a 
uniformly illuminated portion of said focal plane means 
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corresponding to said co-added pixel values comprises a 
fraction of the entire focal plane means such that the 
fraction of the entire focal plane means comprising the 
uniform illuminated portion varies in inverse proportion 
to said exposure time. 

5. The hyperspectral imager of claim 1 wherein said 
focal plane means comprises a charge transfer device 
and said pixel recording means comprise charge packets 
therein characterized by amounts of charge propor- 
tional to an amount of light received by said focal plane 
array at locations coincident with said charge packets. 

6. The hyperspectral imager of claim 5 wherein said 
charge packets are transported at a speed equal to a 
speed at which said scene advances across said focal 
plane array in said longitudinal direction. 

7. The hyperspectral imager of claim 6 wherein said 
charge transfer device comprises N rows of said pixel 
recording means transverse to said longitudinal direc- 
tion, and wherein exposure time tq corresponds to a 
number n q of pixel rows across which said scene ad- 
vances during said exposure time tq, and wherein the 
number of spectral bands M and the exposure times tq 
are constrained by said electronic controller means to 
satisfy 


nl-\-n2+ . . . +/z 9 + . . . + nM<N. 

and wherein N and n q are positive integers. 

8. The hyperspectral imager of claim 7 wherein 
M=N for maximum number of recorded images of said 
scene in different spectral bands. 

9. The hyperspectral imager of claim 7 wherein M = 1 
for maximum exposure time and maximum co-adding of 
said pixel values for maximum signal-to-noise enhance- 
ment of said scene in a single spectral band. 

10. The hyperspectral imager of claim 1 wherein said 
electronic spectral filter means comprises one of a class 
of devices which includes an acousto-optical tunable 
filter. 

11. The hyperspectral imager of claim 1 wherein said 
focal plane array vies said scene through an aperture 
having a fixed slit size. 

12. A method of operating hyperspectral imaging 
apparatus, said apparatus comprising focal plane means 
comprising an array of pixel recording means for re- 
ceiving light from a scene moving relative to said focal 
plane means in a longitudinal direction, said pixel re- 
cording means being transportable at a controllable rate 
in said focal plane means in said longitudinal direction 
by rate-controllable transport means, said method com- 
prising: 

selecting an exposure time tq of said focal plane 
means, and removing pixel recording means ex- 
posed to a scene in an active area of said focal plane 
means from said active area and storing signals 
charges accumulated therein upon expiration of 
said exposure time tq; 

selecting a spectral band q of light received by said 
focal plane means from said scene for each expo- 
sure time tq; and 

recording a selected portion of said scene in respec- 
tive groups of said pixel recording means through 
respective M spectral bands with a respective ex- 
posure time tq for each respective spectral band q, 
wherein M is a positive integer. 

13. The method of claim 12 wherein during said expo- 
sure time tq each pixel recording element is transported 
by m pixel spacings in said longitudinal direction while 
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said scene advances in said longitudinal direction by m 
pixel spacings, whereby pixel values of said scene are 
co-added m times so as to enhance a signal-to-noise ratio 
thereof, and wherein m is a positive integer. 

14. The method of claim 13 wherein t q , M and m are 5 
each different numbers for different values of q, 
whereby different scene portions are recorded in re- 
spective pixel recording means with a respective expo- 
sure time t q for each band q and co-added a different 
number of times m for each respective spectral band q. 10 

15. The method of claim 13 wherein a uniformly 
illuminated portion of said focal plane means corre- 
sponding to said co-added pixel values comprises a 
fraction of the entire focal plane means such that the 
fraction of the entire focal plane means comprising the ^ 
uniform illuminated portion varies in inverse proportion 1 
to said exposure time, and wherein said storing step 
comprises storing only the pixel recording means corre- 
sponding to said fraction. 

16. The method of claim 12 wherein said focal plane 
means comprises a charge transfer device and said pixel 20 
recording means comprise charge packets therein char- 
acterized by amounts of charge proportional to an 
amount of light received by said focal plane array at 
locations coincident with said charge packets. 

17. The method of claim 16 wherein said charge 25 
packets are transported at a speed equal to a speed at 
which said scene advances across said focal plane array 

in said longitudinal direction. 

18. The method of claim 17 wherein said charge 
transfer device comprises N rows of said pixel record- 30 
ing means transverse to said longitudinal direction, and 
wherein each exposure time tq corresponds to a number 

n^ of pixel rows across which said scene advances dur- 
ing said exposure time tq, said method further compris- 
ing constraining the number of spectral bands M and the 35 
exposure times tq to satisfy 

. . . -f- nM~ N. 

and wherein N and n q are positive integers. 40 

19. The method of claim 18 wherein M— N for maxi- 
mum number of recorded images of said scene in differ- 
ent spectral bands. 

20. The method of claim 18 wherein M = 1 for maxi- 
mum exposure time and maximum co-adding of said 
pixel values for maximum signal-to-noise enhancement 4 
of said scene in a single spectral band. 

21. The method of claim 12 wherein said electronic 
spectral filter means comprises one of a class of devices 
which includes an acousto-optical tunable filter. 

22. The method of claim 12 wherein said focal plane 50 
array views said scene through an aperture having a 
fixed slit size. 

23. In an imager comprising focal plane means com- 

prising an array of spaced pixel recording means for 
receiving light from a scene moving relative to said 
focal plane means in a longitudinal direction, said pixel 
recording means being transportable at a controllable 
rate in said focal plane means in said longitudinal direc- 
tion by rate-controllable transport means; hyperspectral 
imaging apparatus comprising: 60 

controllable electronic shutter means for adjusting an 
exposure time tq of said focal plane means, 
whereby pixel recording means in an active area of 
said focal plane means are removed therefrom and 
signal charges accumulated therein are stored upon 65 
expiration of said exposure time tq; 

controllable electronic spectral filter means for se- 
lecting a spectral band q of light received by said 
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focal plane means from said scene for each expo- 
sure time tq; and 

electronic controller means connected to said trans- 
port means, to said electronic shutter means and to 
said electronic spectral filter means for controlling 

(a) said controllable rate at which said recording 
means is transported in said longitudinal direction, 

(b) said exposure time tq and (c) said spectral band 
q so as to record a selected portion of said scene in 
respective pixel recording means through M spec- 
tral bands with a respective exposure time tq for 
each respective spectral band q, wherein M is a 
positive integer. 

24. The hyperspectral imager of claim 23 wherein 
during said exposure time tq each pixel recording ele- 
ment is transported by m pixel spacings in said longitu- 
dinal direction while said scene advances by m pixel 
spacings in said focal plane means in said longitudinal 
direction, whereby pixel values of said scene are co- 
added m times so as to enhance a signal-to-noise ratio 
thereof, and wherein m is a positive integer. 

25. Tlie hyperspectral imager of claim 24 wherein t q , 
M and m are each different numbers for different values 
of q, whereby different scene portions are recorded 
with a respective exposure time t q for each band q and 
co-added in said pixel recording means a different num- 
ber of times m for each respective spectral band q. 

26. The hyperspectral imager of claim 24 wherein a 
uniformly illuminated portion of said focal plane means 
corresponding to said co-added pixel values comprises a 
fraction of the entire focal plane means such that the 
fraction of the entire focal plane means comprising the 
uniform illuminated portion varies in inverse proportion 
to said exposure time. 

27. The hyperspectral imager of claim 23 wherein 
said focal plane means comprises a charge transfer de- 
vice and said pixel recording means comprise charge 
packets therein characterized by amounts of charge 
proportional to an amount of light received by said 
focal plane array at locations coincident with said 
charge packets. 

28. The hyperspectral imager of claim 27 wherein 
said charge packets are transported at a speed equal to 
a speed at which said scene advances across said focal 
plane array in said longitudinal direction. 

29. The hyperspectral imager of claim 28 wherein 
said charge transfer device comprises N rows of said 
pixel recording means transverse to said longitudinal 
direction, and wherein each exposure time tq corre- 
sponds to a number n^ of pixel rows across which said 
scene advances during said exposure time tq, and 
wherein the number of spectral bands M and the expo- 
sure times tq are constrained by said electronic control- 
ler means to satisfy 

rtl+/z2-f- . . . . . . -j- nAf<C_N. 

and wherein N and n q are positive integers. 

30. The hyperspectral imager of claim 29 wherein 
M=N for maximum number of recorded images of said 
scene in different spectral bands. 

31. The hyperspectral imager of claim 29 wherein 
M=1 for maximum exposure time and maximum co- 
adding of said pixel values for maximum signal-to-noise 
enhancement of said scene in a single spectral band. 

32. The hyperspectral imager of claim 23 wherein 
said electronic spectral filter means comprises one of a 
class of devices which includes an acousto-optical tun- 
able filter. 

33. The hyperspectral imager of claim 23 wherein 
said focal plane array views said scene through an aper- 
ture having a fixed slit size. 

***** 



